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ASSESSING NATIONAL DATA ON EDUCATION 



Gauging the condition of education in the Unted States necessarily 
involves some assessment of how well the gauges work. Are our information 
systems measuring key indicators? Do they provide accurate readings? Are 
they optimally linked? In this assessment, we apply these questions of 
coverage, quality, and linkages to the current collection of national stati: 
on education at the preprimary. elementary/secondary, and higher education 
levels. At each level, we discuss fundamental policy issues, summarize 
major data bases, and evaluate the potential of existing data to inform 
policy discussions. In the final section, we recommend ways to improve the 
collection of national statistics on education. 



DA TA ON PREPRIMARY EDUCATION 

Concern about preprimary education in the United States has been stimulated 
by rising participation rates and a recent upturn in the population of preschool 
age. Heightening the concern is conflicting evidence over the importance of 
such education for later achievement. Results from the Perry Preschool 
Project (Berrueta-Clement et al., 1984) and earlier collaborative work (Lazar 
et al., 1974) indicate that preschool experience has sustained benefits for 
disadvantaged children. These results have been used to justify expansion of 
publicly supported preschool education. Critics, however, have argued that 
these findings are not applicable to most preschool programs. 

The heightened interest in preschool education and disagreement over its 
effectiveness have generated a need for information on what education is 
currently being provided to preschoolers, what outcomes should be expected, 
and how preprimary programs complement learning in the home. A number of 
questions have been raised: 

0 Availability . What types of preschool programs are available? 
What types of children participate in various programs? To what 
sxtent is preschool education available to low-income families? 
How are programs supported? What is the mix of public and private 
funds? Is the current pool of preprimary programs adequate to 
meet the demand? Can it meet future demand? 

0 Standards. Under what standards do preprimary programs operate? 
Do standards differ by community, family background of participants, 
and sponsorship? What standards distinguish high-quality programs? 

0 Family-school interaction. How do preprimary programs complement 
learning in the home? How do families, in turn, reinforce preschool 
learning? 
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Only a half-dozen surveys collect nationally representative data relevant 
to these policy concerns. Table 1 shows the six data bases and the categories 
of information they cover. Responsibility for government-sponsored surveys 
is divided among various federal agencies. The Education Department surveys 
that gather some preschool information include the Common Core of Data (CCD) 
on public school systems and the High School and Beyond (HSB) longitudinal 
study of the National Center for Education Statistics, and the National 
Assessment of Educational Progress (NAEP) of the National Institute of Education. 
CCD provides statistics on kindergarten and prekindergarten enrollment in 
regular public school systems and full-time-equivalent teachers at this 
level. HSB and NAEP ask students to report retrospectively on whether they 
had been enrolled in preprimary programs. 

The Bureau of the Census collects data annually on preprimary enrollment 
and occasionally on child-care arrangements of working mothers through the 
Current Population Survey (CPS). The Department of Health and Human Services 
(HHS) data collection on preprimary education is now limited to an annual 
Head Start survey, but in 1976-77 HHS also undertook a comprehensive study of 
day-care centers. Although the 1976-77 National Day Care Study was a one-time 
study, it may suggest a model for future data collections and is included 
among the data discussed here. 

Data Coverage 

Without knowledge of the current availability of preprimary education 
providers, it is difficult to assess how well enrollment demand is being 
accommodated now and is likely to be accommodated in the future. To answer these 
questions, policymakers need information on enrol Imonts in various types of 
preschool programs, costs of services, and access jr different population 
groups to preschool programs. 

National data are available on participation in public and private 

nursery schools and kindergartens from the CPS. Despite the growing interest 

in private sector involvement in providing educational services, however, 

there is no information on the type of provider (whether nonprofit or prof i tmaking) 

and on sponsorship (whether church, employer, or community-sponsored). 

The availability of services to different population groups may hinge on 
costs and the ability to pay for such programs but data are sparse on costs 
for preprimary programs. The National Day Care Study collected fairly extensive 
data on expenditures, revenue sources, fees, donated resources, and staff 
salaries, but these data applies to only one type of preprimary program, 
licensed day care, and the data are a decade old. In response to special 
requests from the Education Department, the CPS asked for information on 
tuition paiQ for private nursery schools and kindergartens in 1979 and 1982. 
But tuition data are not routinely collected in the CPS and can be tied to 
only a few program characteristics. The Head Start program annually estimates 
its average cost per child, but expenditures for Head Start cannot be ge'^eralized 
to other preschool arrangements. Moreover, the Head Start program serves 
only one in four of the eligible low-income children and provides no data on 
the total need for preschool care among low-income populations or on cost 
barriers to providing such care. 
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TABLE 1 : VARIABLE CATEGORIES COVERED BY SELECTED PREPRIMARY EDUCATION 
DATA BASES 



Data bases 



Processes-- 
curriculum. 

Outcomes climate. Resources Student 
standards background 



Education Department 
Common Core of Data (CCD) 
High School and Beyond (HSB) 



IS. 



National Assessment of 
Educational Progress (NAEP) 



IN. 



Other Federal Agencies 

Current Population Survey (CPS) 

Head Start annual survey 

National Day Care Study 
(1976-77) 



IS. 



I_l 



[21 = Complete data 
= Incomplete data 
= No data 
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Questions of standards for preprimary education— and government involvement 
in setting standards--are being discussed in state legislatures across the 
nation. Informed debate requires knowledge of what constitutes adequate 
standards and how well existing services match these standards. Moreover, 
in defining an adequate level of service, it is important to know the 
extent to which preschool experience affects a child's short- and long-term 
progress in school. Questions to be addressed in establishing standards 
include these: What credentials should staff have? What represents an 
acceptable staff-to-child ratio? How much time should young children devote 
to structured learning activities? 

Information on the quality of current programs is virtually nonexistent, 
lacking the most basic data on preschool processes and outcomes. The evidence 
from effective preschool programs points to the importance of the quality 
issue. The Perry Preschool Project, which yiplded long-term benefits for 
young children from disadvantaged homes, stressed the "high quality" of the 
program, as measured by high staff-to-pupil ratios, extensive home visits, 
and a fully articulated curriculum. 

The last national attempt to gather comprehensive information on preschool 
processes was the National Day Care Study in 1976-77. This study identified 
variables that appeared to improve children's gains in test scores, such as 
the specific child-related education of the caregiver, a finding that has 
implications for teacher certification. 

The Head Start program's ongoing annual survey asks for information 
on staff credentials in early childhood education, but does not ask for any 
child performance measures beyond the number of children who drop out of 
the program, thereby missing an opportunity to link standards with outcomes. 

Data from HSB and NAEP also could potentially tie participation in 
preschool with later performance, but retrospective reporting and the fact 
that nothing is known about the characteristics or quality ot these programs 
limit the utility of these data. 

"If studies of school achievement have shown one thing, it is the importance 
of the family," concludes Coleman in his latest study of schools (1982, p. 19). 
To assess the preschool experience, it is critical to discover the complementary 
nature of preschool -home relationships. For example, how much time do parents 
spend participating in learning activities with their children? What home 
learning activities are most beneficial? What are the net effects of preschool 
education in relation to the home environment? 

Only one national data base provides any reasonable information on home 
activities for children. The University of Michigan Time-Use Study gathered 
information in 1975 and 1981 on parent-child interaction through home diaries, 
but this study included too few young children to focus on the preschool 
years ar.d did not collect performance data for this age group. 
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Indeed, no national surveys have looked at the home environment and the 
preschool environment together to suggest how learning may be reinforced or 
confounded by different settings. Some researchers have posited that the 
"hidden curriculum" provided in middle-class homes exerts so strong an influence 
on learning that instruction offered in a formal preprimary setting yields no 
net learning benefits for middle-class youngsters. Current national data 
bases provide no information with which to address this subject. 

Data Quality 

Data availability is obviously the largest problem at the preprimary level, 
but accuracy is also a concern. When the various data bases do overlap, 
they may not agree. For example, public preprimary enrollment reported through 
CCD shows some half-million children fewer than the 3.5 million estimated 
from the CPS household survey. 

An examination of the CCD suggests that some local school districts ^nd 

state education agencies have difficulty reporting accurate preprimary enrollment 

and staffing information, particularly in distinguishing between headccunt 

and full-time-equivalent enrollments. Yet household respondents in the CPS 

may be equally inaccurate in reporting preprimary enrollment. Although 

the CPS distinguishes between home day care and formal preprimary programs, 

it leaves the classification of center-based day care to the household respondent. 

Thus, the rather thorny problem of defining center care as instructional or 

custodial is left to the respondent and presumably this definition could vary 

from one respondent to the next. 

Data Linkages 

A major hindrance to progress in developing data on preprimary education 
is the fact that no single agency has a clear mandate to collect the information. 
The Department of Education is just beginning to formulate proposals to 
address the area of early childhood education. The largest federal education 
program at the preprimary level. Head Start, is administered outside the 
Department of Education by HHS. In recent years, HHS has all but closed down 
its statistical collection activity in preprimary education, except for 
annual reporting on Head Start. The comprehensive but costly ($7 million) 
National Day Care Study in the late 1970s required a commitment of resources 
that are no longer available. With no agency yet assuming the lead in this 
area, data collection efforts have been sporadic, piecemeal, and uncoordinated. 



DATA ON ELEMENTARY AND SECONDARY EDUCATION 

Concerns with improving education dominate issues in elementary and 
secondary education. Various reports have called attention to performance 
declines over the recent past and the need to upgrade American education, 
spurring many states to pass costly reforms. Policymakers, educators, and 
the public need to be able to assess whether these reforms are working and 
whether they are getting their morey's worth. In addition, the drive toward 
excellence has heightened the concern about provi ding'equal opportunity to 
all students. It is generally recognized that if the Nation's schools are to 
improve, everyone should be encouraged to share in this improvement. Current 
policy debates revo.^e around these questions: 

? 

Er|c 



0 Educational improvement s Are educational reforms actually improving 
student outcomes? 



0 Resource effectiveness . Are resources being applied to promote 
cost-effective educational proyrams? 

0 Equity , Are the needs of disadvantaged and other special population 
groups being met? 

Although a number of national data bases describe different aspects of 
elementary and secondary education, three that have already been mentioned 
stand out as the major ongoing sources of national information, as shown in 
Table 2. The CCD survey of all state educational agencies and local school 
districts, is the primary source of information on schooling inputs--enrollment, 
finances, and staffing. NAEP focuses on measuring educational outcomes. The 
HSB longitudinal survey is the only one with extensive information on family 
background, school process variables, and student outcomes. Specialized 
information on elementary and secondary education is available from various 
other NCES surveys: the Private School Survey; the Survey of Teacher Demand 
and Shortages; and the Library/Media Center Survey. 

With respect to particular populations of special concern to the federal 
government, the Office for Civil Rights surveys districts and schools for 
information on programs, disciplinary actions, and gradu'iny class composition. 
Individual federal education programs, such as those servirK the handicapped 
or the limited-English proficient, conduct their own surveys of participants 
and coverage. 

Other federal agencies and private organizations also provide elementary 
and secondary education data. The CPS, mentioned earlier, supplies annual 
data on enrollment, public and private, and educational attainment. The 
Justice Department collects information on school discipline. HHS funds 
"Monitoring the Future," an annual survey of riigh school seniors which focuses 
on student attitudes and drug abuse. The National Science Foundation (NSF) 
conducts studies on jt th and science education. The Labor Department's 
National Longitudinal Survey--Youth Cohort (NLS--Youth) contains extensive 
information on family, schooling, and work history. A private consortium, 
the International Association for the Evaluation of Educational Achievement 
(lEA), periodically assesses comparative student performance in various 
nations. The National Education Association (NEA) collects information from 
its state coordinators on enrollment, staff, and salaries. 

Data Coverage 

As a result of the clear demand for school improvement and the expenditure 
of billions of dollars on educational reforms, information is critically 
needed on whether these reforms are improving outcomes and which reforms seem 
most effective. Ultimately, data should indicate whether student performance, 
as measured by educational achievement and attainment, has improved and for 
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TABLE 2: VARIABLE CATEGORIES COVERED BY SELECTED ELEMENTARY AND SECONDARY 
EDUCATION DATA BASES 



Data bases 



Outcomes 



Processes-- 
curriculum, 
c1 imate, 
standards 



Resources 



Education Department 

Common Core of Data (CCD) 

National Assessn.ent of Educational Progress 
(NAEP) 

High School and Beyond (HSB) 

Private School Survey 

Teacher Supply and Demand Survey 

Library /Media Center Survey 

Civil rights survey 

Special education program surveys 

Language minority surveys 

Other Federal Agencies 

Current Population Survey (CPS) 

Justice Department surveys 

National Science Foundation (NSF) surveys 

National Longitudinal Survey--Youth Cohort 
(NLS— Youth) (Labor Department) 

Monitoring the Future--Drug Abuse 
(Health and Human Services) 

Private Organizations 

International Association for Evaluation 
of Educational Achievement (lEA) 

National Education Association (NEA) 

| ~ | = Complete 

= Incomplete data 
= No data 



IZI 



1^ 



IN. 
1^ 



IN. 
fN, 



fNs 



Student 
backgrour 



IN. 
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whom. This information should be available nationally, but also disaggregated 
at least down to the state level, since the states are primarily responsible 
for education. 

Most of the national data bases listed in Table 2 provide little help in 
answering these questions. Few provide performance information. The extensive 
data collected on resources can report little about the quality of inputs, 
such as the quality of teachers. A few data bases contain some information 
on schooling processes, but the data often apply to highly specific areas such 
as the provision of special education for the handicapped. 

HSB presents reasonably good infomation on school outcomes and processes, 
but its performance test is limiteo to only a few items. The information 
pertains only to high school processes and tells us little about experiences 
before high schuol. The national data are of little help to particular 
states working to evaluate their systems. 

NAEP offers some advantages over HSB in assessing reforms. NAEP's 
tests are much more extensive than MSB's and are not limited to high school 
students. N^EP also surveys the schools more often than HSB. Furthermore, 
although NAEP initially focused on outcome data, it has expanded its information 
on classroom processes and school and teacher characteristics, and it contains 
some home background information. 

But NAEP also has disadvantages. First, NAEP is subject specific, that 
is, in a given year, NAEP focuses primarily on a few learning areas instead 
of providing a comprehensive picture of what is happening within the whole 
school. Second, NAEP consists of repeated cross sections and cannot be as 
informative as HSB with respect to measuring transitions over time, such as 
dropping out of school. Furthermore, like HSB, NAEP cannot provide state- 
or district-level comparisons. The current NAEP design does allow s\:ates to 
expand the sample to provide representative scores, but to date only three 
states have done so. Thus, although states carry the primary responsibility 
for education, the nation has no way of accurately assessing how well state 
governments are carrying out these responsibilities. 

Other data bases provide additional information on reform, but all are 
limited in important respects. lEA com( ares achievement in a number of countries, 
but the data are highly aggregated and the long intervals separating reports 
(sometimes 15 years) makes lEA unsuitable as a gauge of reform effects. 

On the question of resource effectiveness, research findings have shown 
that overall expenditures and student performance are only weakly associated. 
Hence, to examine the quality of resources and the ways they transla\:e int'- 
school processes, statistics must be collected below the level of broad 
expenditure components. 
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The CCD annually collects revenue, expenditure, and debt-f manciny statistics 
on some 16,000 school districts. Some of these data are important for assessing 
efficiency issues, such as breakouts of administration and instructional 
components of expe.iditures. but many of the items hardly ^^arrant reporting 
each year. On the other hand, despite the concern over the quality of teache. s- 
the most critical resource, CCD no longer collects information on what each 
state pays its teachers. 

Because administrative records on inputs are readily available, data are 
often collected without sufficient regard to priorities. Library statistics, 
for example, are able to provide data on a full range of variables. Although 
libraries are essential for education, something is wrong when NCES can report 
periodicals' costs but not teachers' salaries. 

Measuring how well groups with special needs, such as language and racial 
minorities, are faring as a result of reform efforts requires performance 
information for these groups and some understanding of the relationship 
between education in school and conditions outside school. Data on special -need 
populations and their performance are available from program data, HSB, and 
NAEP. but the data are inaccurate. And we lack the information to understand 
why children with special needs fall so far behind in school, despite evidence 
of progress in the earliest grades. If we are to measure the extent to which 
students fall behind early, we need detailed information on the school and 
outside environment in the formative pre-high-school years, data that 
are almost nonexistent. 

Data Quality 

The quality of data on elementary and secondary education varies greatly. 
Performance, as indicated by standardized tests, is one of the better-measured 
variables. NAEP, in particular, has devoted considerable effort to improving 
outcome measurements and has introduced a new scaling system that may permit 
comparisons across grades. 

Estimates of high school dropouts are less satisfactory. In the drive to 
promote educational excellence, the high school dropout issue was initially 
swept aside. This appears to have been a serious mistake; evidence from 
several sources shows that the problem is severe. However, the reported 
high school dropout rates are error prone. Estimates from alternate 
sources differ on the extent of the problem and, in the absence of agreement 
on a baseline number, it is hard to gauge whether the dropout problem is 
improving or getting worse. 

A dropout rate can be calculated from the CCD by comparing the residual 
of the number of high school graduates with the number of students in the 
9th grade 4 years previously; this approach yields a dropout rate of 27 
percent. In contrast, the CPS, which uses household interviews for 
information on educational attainment, reports a dropout rate of only lb 
percent. The CPS does not, however, report how schooling was completed, 
whether through graduation from a regular high school program or other 
rr ans, such as earning an equivalency certificate or external degree. Ihe 
dropout rate can also be calculated from the HSB longitudinal study, but 
this rate is generally acknowledged to represent an undercount because it 
misses students who leave school before the end of their sophomore year. 

li 
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School process variables pose similar problems; even the simplest are 
sometimes hard to measure. HSB, :o its credit, compared student self-reports 
of courses and yrades with coursework transcripts and found substantial 
mis^^eporting. Unfortunately, quality control studies have not been carried 
out tor other types of process data, such as information on school climate 
and order. Students' responses need to be validated against reports from 
teachers and administrators on these critical variables. 

Data on the use of resources are so unreliable that assessing efficiency 
in American schools is quite problematic. Larr , implausible year-to-year 
variations in per pupil expenditures appear fo" some states. Pupil-to-staff 
ratios calculated from state reports also are unreliable; for example, the 
highest and lowest state pupil-to-all-staff ratios differed by 140% 
on this measure in 1983. Either the reported numbers are faulty, or some 
states have remarkable efficiency advantages over others. 

On the question of whether special populations are being well served by 
American education, the data are also suspect. A prerequisite for addressing 
this issue is agreement on the target populations; numbers for most of 
the federal target groups are at best questionable. For example: 

0 In 1984, special educational students as a proportion of otate 
enrollment ranged from a low of 5% to a high of 13% (U.S. 
Department of Education, Office of Special Education and 
Rehabilitative Services, 1985). There is no physiological explanation 
that could account for these report differences exceeding 100% 
in the prevalence of handicapping conditions. 

0 The Office of Bilingual Education and Minority Languages Affairs 
estimates that about 3.6 million language-minority children need 
bilingual services, but a recent study based on Census Bureau 
data concludes that two-thirds of the children included in this 
number use English as their usual or sole language, and thus could 
hardly be classified as candidates for instruction In a language 
other than English. 

Finally, there is the difficulty of obtaining educationally relevant 
information on households, such as family income and parental reinforcement of 
education. This information appears to be far more accurately reported 
through home questionnaires rather than from student responses. HSB 
validity studies show that students may seriously misjudge family background 
characteristics; comparisons between students' and parents' reports of 
parental occupation, family income, and mother's work, for example, 
yielded validity coefficients of only about .5 (U.S. Department of Education, 
National Center for Education Statistics, 1984). 

Statistical gathering by the federal government often involves inordinate 
time lapses between data collection and dissemination. Indeed, private 
organizations routinely publish similar data that are both more comprehensive 
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and more timely. For example, the CCD provides the location and enrollments of 
all elementary and secondary schools, but the most recent school year for which 
this information is available is 1982-83. In contrast, private firms have 
compiled similar files that provide much greater information on each school 
site for the 1984-85 school year. As a second example, the NEA regularly 
publishes state-level financial and staffing data for the preceding and current 
school years. These data are generally more complete than those the Education 
Department publishes and at least 1 to 2 years more current. 

Data Linkages 

The collage of elementary and secondary aata bases provides cogent 
examples of the advantages of integrating performance data with other kinds 
of information. Two of the most successful information activities in the 
elementary and secondary school area, in terms of use and attention to 
findings, are HS^ and the Education Department's "wall chart" which graphically 
profiles state-by-state comparisons on college entrance test scores and 
resource variables. By contrast, detailed financial statistics reported in 
isolation in the CCD or extensive library statistics tell little about 
school quality and, hence, ere not widely used. 

The data on teachers also points to the need to consider data collection 
as a cohesive whole. Five separate questionnaires from the Department of 
Education have recently sought information from the states about teachers: 
CCD, HSB, NAEP, Survey of Teacher Demand and Shortage, and a new Public School 
Survey. Yet after all these surveys, the Department still does not know how 
much each state pays its teachers. 

DATA ON HIGHER EDUCATION 

Education Secretary William J. Bennett has called for a rethinking of 
higher education--bot.. its funding and its functions. Questions have been 
raised concerning access, cost, quality, and the role of higher education 
in the United States today. Because student aid accounts for nearly half 
of all the Department of Education's spending, the government needs to know 
If the aid is being properly targeted and wisely spent. Institutional 
efficiency is also being questioned in relation to rapidly rising tuitions 
and purported decreases in faculty teaching loads. Indications of declining 
performance and unbalanced curriculums, moreover, call into question the very 
quality of higher education. The trend toward greater vocational ism in 
college studies, along with increased corporate training, stimulates concern 
that higher e&ucation is not fulfilling its mission. 

Data sources presented in Table 3 are consiaered in this section as they 
pertain to these issues: 
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0 student aid . Who receives federal student aid and how can aid 
be refocused to provide oreater access to the most disadvantaged 
students? 

0 Efficiency . Why have tuition charges increasea faster than 
the inflation rate, and how can escalating costs be contained? 

0 Value. What is the quality of the educational experience and of 
college graduates? Wha^c is being learned? 

0 RoTe . What is the role of higher education, particularly in 
retation to occupational and corporate training? 

Information on higher education is obtained by a dive-^se group of government 
and private organizations. At tne federal level, the Higher Education General 
Information Survey (HEGIS) of NCES collects information from all institutions 
of higher education on enrollment, institutional :ontrol and finance, degrees 
conferred, and faculty employment and salaries. Other information is obtained 
by a series of surveys also sponsored by NCES; these include the National 
Longitudinal Study of the High School Class of 1972 (NLS), HSB, and the Survey 
of Recent College Graduates. In an October supplement the CPS collects data 
on college enrollment by control of institution (public or private), and 
family demographic and social characteristics. NSF surveys gather data on 
scientific and engineering education. The Labor Department's longitudinal 
study (NLS--Youth) tracks the work and education experience of young 
adults. 

Sometimes specific information can be obtained only from sources outside 
of the government. The College Board's Annual Survey of Colleges, for example, 
is a comprehensive survey that queries institutions on admissions policies, 
student charges, standards, and fields of study. The American Council on 
Education (ACE) conducts approximately six Higher Education Panel (HEP) 
surveys each vear on topics of special interest. The National Institute on 
Independent Colleges and Universities (NIICU) surveys student aid in private 
co.ieges; the National Association of State Colleges and Universities collects 
an analogous public college survey. The Cooperative Institutional Research 
Program (CIRP) surveys first-time freshmen, while the National Association of 
College and University Business Officers (NACUBO) quantifies endowment levels. 
As further examples, the American Association of University Professors (AAUP) 
and the Council for Financial Aid to Education (CFAE) report faculty salaries 
and voluntary support to higher education, respectively. 

Data Coverage 

The Reagan administration's 1986 budget recently proposed modifications in 
the federal student aid programs to better target benefits to the neediest 
students. Documenting the trends in aid recipients, though, has proved difficult 
and controversial. The problem is not that the information is insufficient, 
but that different sets of data are disjointed and cannot describe the cotal 
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student aid package in relation to students' financial need. Once again, 
the problem is linkage rather than coverage. 

The efficiency of the higher education production process is another 

issue. In recent years, college tuitions have risen faster than the inflation 

rate. Has efficiency in the production of higher education changed? Information 

to address this question would certainly include data on credit hours taught 

per faculty meniuor, time divided between research and teaching, and class 

size and unit costs of different course offerings. 

HEGIS, the principal federal data base to address these questions, collects 
data on number of faculty members and salaries, enrollment, revenue, and 
expenditures. Yet HEGIS data do not help us answer questions of efficiency. 
Faculty salaries aro not reported according to the amount of time spent on 
research, as opposed to teaching. And information is not provided on credit 
hours taught or on class size from which unit costs by course offering could 
be calculated. 

HEGIS fiscal data are aggregated at the institutional level, a practice that 
precludes reporting resource allocations among academic departments and 
levels. The extent to which costs, for instance, in the sciences are subsidized 
by humanities departments, or graduate programs by undergraduates, is not 
available. In addition, although it is generally assumed that the expansion 
of programs and course offerings has been costly ana inefficient, no figures 
are available with which to measure the expense of offering a multiplicity of 
courses. 

Alternative sources for certain information about higher education inputs are 
available, but little is known about the efficiency with which resources are 
allocated. The AAUP publishes annual salary statistics for college faculty 
by rank but reports nothing on teaching loads. It has been suggested (Noah, 
1985) that teaching loads have been cut substantially over the past four 
decades, yet this claim cannot be corroborated because no national survey 
divides faculty time among teaching, research, and other endeavors. 

There have also been recent expressions of concern about the value and 
diversity of the college course of study. "Although more than 50 percent of 
America's high school graduates continue their education at American colleges 
and universities," writes Secretary Bennett (1984), "few of them can be 
said to receive there an adequate education in the culture and civilization 
of which they are members." Information on course enrollments by subject area 
and on the extent of remediation would be helpful. Test scores would indicate 
what students actua'^ ly learned in courses. The apparent growth in remediation 
would suggest a "dumbing down" of coursework in colleges. Although detailed 
information is available about the major fields of graduates, little is known 
about the courses graduates took outside their fields. For example, HEGIS 
reports the number of men and women receiving degrees in home management, but 
not the number of courses taken by students who major or do not major in the 
humanities. ACE surveys indicate total credit hours in selected science and 
humanities programs, but the data do not permit translation of these credit 
hours to individuals. 
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TABLE 3: VARIABLE CATEGORIES COVERED BY SELECTED HIGHER EDUCATION DATA BASES 



Student 

Data bases Outcomes Processes Resources background 

Education Department 

Higher Education General Information 

Survey (HEGIS) — |_| 

National Longitudinal Study of the High 
school Class of 1972 (NLS) " - 



High School and Beyond (HSB) |N, — — |_| 

Survey of Recent College Graduates fN^ — — 

Other Federal Agencies 

Current Population Survey (CPS) |\, — ' " 

National Science Foundation (NSF): Surveys 
of Science and Engineering Expenditures, 
Federal Support, Personnel, and Graduates — 

National Longitudinal Survey--Youth Cohort 

(NLS--Youth) (Labor Department) | | fS^ — |_| 

Private Organizations 

National Institute on Independent Colleges 

and Universities (NIICU): Survey of 

Student Aid Recipients ~ fN^ | | 

National Association of State Colleges 

and Universities (NASCU): Survey of 

Student Aid Recipients ~ | | 

College Board: Annual Survey of 

Institutions | | fX^ 

Cooperative Institutional Research 
Program (CIRP): American Freshman Norms [N^ 

American Council on Education (ACE): 
Higher Education Panel (HEP) — fN^ — 

National Association of College and 
University Business Officers (NACUBO): 
Comparative Performance Study and 

Investment Questionnaires ~ fX^ 

American Association of University 
Professors (AAUP) — [X^ 

Council for Financial Aid to Education (CFAE) — f\ 



ERLC 



I I - Complete data " Incomplete data = Blank 

421 



16 



Data that would quantify the returns to a college education in terms of 
increased knowledge also are lacking. No uniform test, such as the Graduate 
Record Examination, is given to a representative sample of graduates to test their 
general knowledge. One study of changes over the past two decades found 
significant declines in graduate admissions test scores in most fields 
(Adelman, 1984), but these results were not drawn from statistically 
representative sample of all graduates, because only those intending to yo to 
graduate school took the exams. 

Of concern is the quality of graduates produced by various programs and 
in different fields, particularly graduates of teacher education programs. 
Beyond the NLS study of members of the 1972 high school class who went into 
teaching, little definitive is known about the quality of graduates prepared 
to teach. The Survey of Recent College Graduates asked for grade-point 
averages, but this measure is a poor indicator of quality and no substitute 
for college transcript data. College transcripts would give not only grades 
but actual courses that would suggest the extent to which teaching graduates 
are fully prepared in their subject matter or were enrolled in watered-down 
courses. 

Concern has also arisen over the mission of higher education in relation 

to other providers of postsecondary training. Some observers charge that 

higher education, particularly in community colleges, has become indistinguishable 

from vocational training. Conversely, corporate training is being provided 

in areas traditionally reserved for universities and colleges, "liese 

developments suggest the need for information about the extent of the 

overlap and the relationship between training provided by businesses and 

postsecondary institutions. For that matter, the academic-occupational mix 

in college programs remains an important unknown, critical to assessing 

higher education's role in relation to the role of other postsecondary 

providers. 

Data Quality 

Although many data sets provide information about the specific policy 
issues raised here, multiple sources often result in conflicting and incomplete 
overall higher education policy evaluations. Both the HEGIS and CPS, for 
example, report enrollment, but tabulations differ even for this most basic 
statistic. The two surveys basically agree on total enrollment but show 
discrepancies by ful 1 -time/part-time status and 4-year/2-year disaggregations 
and wider differences by graduate/undergraduate breakdowns. For example, CPS 
estimates graduate school enrollment to be one-third larger than the figure 
reported through HEGIS, although in other categories CPS finds fewer students 
than HEGIS. 

Although HEGIS remains the federal government's primary instrument for 
monitoring higher education, the periodicity of many of these surveys has 
been interrupted in recent years. Because of processing delays and technical 
difficulty, NCES has abandoned attempts to release the 1982-83 data on faculty 
salaries and some of the financial and degrees data. Even in years without 
abnormal delays, HEGIS data were not available to analysts until approximately 
two years after the surveys were taken. 
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Data Linkages 



\ major problem associated with higher education data is that information 
from multiple sources cannot be combined to form a complete picture. The 
relationship between student aid and financial need provides an example of 
the lack of linkage among data sets and the difficulty this causes to analysts. 

Any evaluation of student aid programs requires an understanding of the 

income distribution of students' families and the types and amounts of aid 

these students receive. These are important policy considerations, especially 
given the debate over proposed federal funding reductions. 

The fundamental problem with student aid information is that it is collected 
for administrative purposes by those responsible for each program. Hence, 
coverage may not be a problem, but organization is. Because the information 
is generally not consolidated into files that cover all student aid programs, 
it is impossible using federal data bases, to determine how the need for 
student aid is distributed and how federal aid combines with other revenue 
sources to meet this need. 

Two private sources of student aid information, however, cut across programs 
and could provide the needed data. Identical questionnaires concerning 
student aid are sent to both public and private institutions for the National 
Association of State Colleges and Universities (NASCU) and the National 
Institute on Independent Colleges and Universities (NIICU), respectively. 
These surveys could, if merged, provide the information needed for policy 
analysis and discussion. The Education Department can obtain the NASCU 
public collage data, but NIICU will not make available the private college 
data tapes. 

The problem, again, is that much o^ the information useful for analyzing 
the higher education process is piecemeal, serving only the specific needs of 
the originating agency. Some national coordination for these blocks of 
data would vastly improve the information base. 



RECOMMENDATIONS FOR IMPROVING NATIONAL DATA 

The tremendous national interest in educational improvement provides the 
impetus for reform of our national base of educational statistics, but the 
task will not be easily accomplished. Inertia and special interests 
will work against developing a coherent data collection strategy. Policymakers 
may De unwilling to wait for real improvements. 

Given these problems, identifying a clear agenda becomes all important 
for improving national statistics. Extensive analysis is needed before a 
complete set of reforms can be specified. Analysts must carefully investigate 
the design options for data collection before final decisions on implementation 
are made. Resource and political constraints also will impose trade-offs 
among options. Our purpose in setting forth this agenda is to identify the 
most important directions for reform. We have divided our recommendations 
along the three types of evaluation criteria used throughout this paper: 
data coverage, data quality, and data linkages. 
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Data Cover jge 



The federal government needs to ask the important questions and collect 
data accordingly. It already collects a considerable amount of educationally 
related data, but some of the data now collected may have relatively little 
policy value, whereas other data not now gathered may have high informational 
value. 

Recommendation 1: Eliminate low priority data items . 

The continued collection of data with little policy worth cannot be 
justified. A comprehensive reconsideration of survey instruments requires 
a item-by-item analysis, but various candidates for paring are evident. 
They include the following: 

0 Annual HEGIS collection of degrees awarded by sex, for six- 
digit degree codes at the subbaccalaureate level that include 
facts such as the number of female degree recipients in fashion 
merchandising or jewelry marketing; 

0 Library statistics on book binding costs and number of 
audiovisual materials; and 

0 Annual CCD collection of school district finances detailing 
expenditure and revenue statistics for all 16,000 school districts. 

Recommendation 2: Identify and fill information gaps on a priority basis in 
areas of major policy interest. 

Some gaps in the data can be filled by adding items to existing surveys 
without incurring substantial costs, and these efforts should proceed. 
Examples include: 

0 Adding questions on preprimary education to the CPS to 
differentiate provider types and extending coverage 
of this item to estimate the number of children in 
home day care; 

0 Requesting teacher salary data by state as a regular part of the 
CCD; 

0 Including on the CPS a question on how high school was completed, 
whether through graduating from a regular 4-year program or by 
earning an equivalency certificate or external degree; 

0 Adding questions to HEGIS about number of faculty hours spent 
teaching as a rough measure of instructional load; and 

0 Adding a college transcript collection to the Survey of 
Recent College Graduates. 
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Supplementing existing surveys will not, by itself, produce much of the new 
data needed to resolve many of the most glaring deficiencies in policy 
information needs. These deficiencies include the following: 

0 Preprimary education . Little is known about the organization 
and support for preprimary education, the nature or quality 
of these programs, and the relationship between preschool and 
home learning. 

0 Elementary school . No recent study describes how course 
content relates to outcomes. Of special interest at this 
level would be information on the use and effectiveness of 
alternative teaching approaches to development of reading and 
mathematics skills. NAEP may begin to yield information on 
coursework in the 4th grade, but additional data are needed to 
describe how learning at this level relates to learning before 
and after this grade. 

0 Junior high school . Two areas deserve special attention: 
First, the adolescent years are or.3S in which behavioral 
problems of students become serious. Many youths may drop 
out of school at this level, yet national data about attrition in 
junior high is lacking. Information on how discipline and 
dropout problems develop may require longitudinal data collection 
at the junior high leve'.. Second, deficiencies in higher-order 
thinking skills surface in adolescence, and these deficiencies need 
to be related to course-taking and schooling processes. 

0 Hic|her education student aid . The primary problem is the absence 
of a file for each student that shows aid from all sources in 
relation to need. This consolidated record is essential for 
analyzing the effects of student aid reforms. 

0 College learning . Except for the number of degrees granted and 
the unrepresentative graduate admission exam scores, there U 
little or no information on the learning that takes place in 
college or even on the distribution of coursrwork. 

This rather long list unfortunately reflects the sorry condition of current 
national statistics. In view of financial and staff limitations, priorities 
must be established. Attention must be paid to state-of-the-art problems m 
developing designs that yield reasonably cost-effective information payoffs. 
Examples of the difficulties involved in conducting surveys include 
the following: 




20 



425 



0 Preprimary education occurs in myriad settings, the most important 
of which may be the home. The universe of settings is poorly defined; 
master lists to draw samples often do not exist. Distinctions between 
nursery, kindergarten, preschool, and day care may be only nominal. 
Studying classroom processes at the preprimary level may require the 
use of detailed and costly observational techniques. 

0 Previous attempts to measure schooling processes at the elementary 
level have yielded little useful information. The Sustaining 
Effects Study, a $20 million longitudinal study in the mid-19/Os, 
thoroughly examined elementary schools, yet told very little about 
effective elementary schools and classrooms. For that matter, it 
even failed to achieve its primary objective of settling the debate 
over the long-term effectiveness of compensatory education programs. 

0 Colleges, in particular, and postsecondary institutions, in general, 
are extremely diverse, and it is not at all clear how to measure 
outcomes for these institutions. 

In view of these methodological and definitional considerations and 
differing policy priorities, experts will disagree over subjects to pursue. 
Among this list, the two activities we recommend for highest initial priority 
are these: (1) the junior high school study, because schooling in these 
grades may be the key to rejuvenating the high school and because so 
little is known about processes at this level; and (2) the student aid 
study, because this issue is of extreme immediate importance. In other 
areas, conceptual and measurement efforts are clearly needed and should 
begin at once. For example, efforts to separate educational from custodial 
services at the preprimary level and to develop of appropriate measures 
of learning at the college level should be pursued. 

Data Quality 

Inaccuracies and inconsistencies in Education Department statistics pose 
serious problems. Once data are published, they become accepted as fact, 
regardless of the caveats that may accompany their publication. In some 
instances, the reader is not even warned of data weaknesses. These concerns 
over data quality are not new ones, but seemingly they arise, year after 
year, with little improvement. The Education Department should immediately 
begin to address the problems of improving the adequacy of its data. 

Recommendation 3: Establish an office of quality control . 

No such office now exists within the Education Department, nor has any 
office promulgated a set of standards to ensure adequate data quality. An 
office that has this function as its primary responsibility is essential 
to the job of inproving data quality. This office should be independent of 
the data-collecting offices and should oversee data collection of both general 
purpose statistics and program data. 
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Recommendation 4: Give highest priority to impfov ipr the auaTitv of eTementary 
and secondary education data. 

Although all information must be subject to quality control reviews, the 
most glaring accuracy problems relate to elementary and secondary education 
data. The states must be held accountable for providing the Department of 
Education with consistent and accurate data. 

The failure of certain data to pass muster when subjected to external 
validity checks must be immediately addressed. In particular, differences 
between NCES and Census Bureau data on estimated dropout rates and private 
school attendance can no longer be ignored simply because of organizational 
divisions at th*? federal level. The two organizations should immediately 
establish a joint review group to assess the reasons for inconsistencies in 
the data each collect. 

Recommendation 5: Modernize procedures for data collection . 

NCES has had problems of providing timely data, especially with some of 
its larger surveys such as CCD and HEGIS. Several types of reforms could 
improve the currency of information. Using the telephone instead of a mailed 
questionnaire would speed collection of school universe data. When there 
are a limited number of r^-pondents, such as 50 states, computer networks 
seem to be a sensible approach. Once data are collected, they should not 
languish. Analysis contracts should be built into the overall data collection 
effort to expedite reporting of the data. 

Data Linkages 

The number of distinct data collection instruments related to education is 
quite large. The approximately 35 data bases reviewed in this study are only a 
partial listina. These activities lack cohesi veness. Although a master 
plan relating all data collection efforts would be quite unwieldy, far too 
little attention is now paid to the advantages that could be derived from 
coordinating or combining related efforts. 

No one reason explains why coordination of national data-gathering activities 
has not progressed. Sometimes the reasons are historic. New data collections 
were planned for specific purposes and the planners paid inadequate consideration 
to whether these purposes could be better met through existi ng acti viti es. 
Moreover, data needs may be defined so narrowly that the benefits of an 
integrated data set are not perceived. At other times, the problem is 
bureaucratic. It is unnatural to expect one statistical division or agency 
to transfer its responsibility to another. Whatever the reason, the limited 
federal funds available for education statistics make coordination of data 
activities imperative. 

Steps the Education Department should consider to strengthen linkages among 
its statistical activities include the following: 
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Recommendation 6: Investigate the feasibiTity of linking the most costly data 
collection systems in the Department of Education, 

Longitudinal data and performance assessraents are acknowledged to be 
high-budget items. If we can detach ourselves from the ways data systems are 
currently organized and administered, we may see some alternatives for collecting 
information. Take, for example, the NAEP. Many would say that NAEP has had 
limited usefulness because it could not be linked directly to school policies 
and practices. Prior to the 1983-84 assessment, performance measures could 
be tied to only a few student characteristics and to no curriculum or process 
variables. The recent release of preliminary NAEP data suggests that the 
inclusion of fairly comprehensive student and teacher questionnaires will 
prove most valuable. 

Other tie-ins to major outcome assessment activities should be considered. 
In particular, the question should be raised as to whether the distinctions 
between repeated cross-sectional studies, such as NAEP, and longitudinal studies, 
such as HSB, are real or artificial. For example, attaching a small longitudinal 
component to the NAEP 7th grade sample might provide a measure of the extent 
of attrition at this early level. The relationship among longitudinal surveys 
also should be considered, such as, for instance, how NCES longitudinal 
surveys could be coordinated with the Labor Department's NLS--Youth survey 
which also obtains longitudinal cohort data. Both HSB and Labor's Youth 
Cohort have transcript studies that, coincidental ly, are being directed 
by the same contractor. 

Recommendation 7: Initiate informal discussions with representatives of 
Departments of Education, Health and Human Services, and Labor, and the 
Bureau of the Census to coordinate data collections . 

Data collectors in the private sector also should be brought in to the 
discussions. 

Recommendation 8: Consider, when developing questionnaire items, those variables 
that have been shown to be most important to educational outcomes . 

Statisticians and survey monitors should draw upon recent school effectiveness 
studies to help frame survey instruments. 

Recommendation 9: Establish an education data bank to improve survey consistency 
across data bases and over time. 

This data bank would pool items related to education within and outside 
government and might include items used in state and local surveys and case 
studies. 

Recommendation 10: Keep expectations high . 

We might take a cue from recent research on HSB which shows the power of 
positive thinking and persistence on student performance. Despite limited 
funds, national data collections can be improved. Because funds are limited, 
greater efficiency it is important to promote in our information systems. In 
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addition, data producers should call attention to and take credit for the 
policy payoffs from study findings. 

In addressing 1985 college graduates, Secre -^v Bennett offered some 
sound advice: "It is practical optimism that recommend/* As we consider 
reforming national statistical collections, must think practically 
and optimistically about the task ahead. 
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